BAYES'S FORMULA

Tags: #◇Computer Science #◇STT101A #◇Conceptual #◇Bayes's Formula #◇Probability #◇Statistics

---

◇Overview

Bayes' Formula (Bayes' Theorem) provides the mathematical mechanism for updating conditional probabilities in light of new observed evidence. It connects the prior probability of an underlying cause with the posterior probability after an observable consequence occurs, formalizing rational Bayesian inference across machine learning, cybersecurity threat detection, and statistical modeling.

◇Key Definitions

• Prior Probability ($P(B_j)$): The initial probability of hypothesis or state $B_j$ before observing new evidence.
• Likelihood ($P(A \mid B_j)$): The conditional probability that evidence $A$ is observed, assuming hypothesis $B_j$ is true.
• Marginal Probability / Evidence ($P(A)$): The total probability of observing evidence $A$ across all mutually exclusive scenarios in the sample space.
• Posterior Probability ($P(B_j \mid A)$): The updated probability of hypothesis $B_j$ given that evidence $A$ has actually occurred.
• Partition of Sample Space: A collection of mutually exclusive and collectively exhaustive events $\{B_1, B_2, \dots, B_k\}$ such that $B_i \cap B_j = \emptyset$ for $i \ne j$, and $\bigcup_{i=1}^k B_i = S$.

---

◇Theoretical Framework: Derivation & Mathematical Formulation

The Law of Total Probability

For any event $A$ and a partition $\{B_1, B_2, \dots, B_k\}$ of sample space $S$:
$P(A) = \sum_{i=1}^k P(A \cap B_i) = \sum_{i=1}^k P(A \mid B_i) P(B_i)$

Bayes' Theorem

By the definition of conditional probability:
$P(B_j \mid A) = \frac{P(B_j \cap A)}{P(A)}$
Substituting the multiplication rule $P(B_j \cap A) = P(A \mid B_j) P(B_j)$ into the numerator and the Law of Total Probability into the denominator yields:
$P(B_j \mid A) = \frac{P(A \mid B_j) P(B_j)}{\sum_{i=1}^k P(A \mid B_i) P(B_i)}$

Probability Tree Model

---

◇Stepped Numerical Walkthrough: The Base Rate Fallacy in Threat Detection

Scenario: An enterprise Intrusion Detection System (IDS) monitors encrypted network flows for a rare zero-day rootkit.

• Base Rate (Prior Prevalence): Only $0.5\%$ of network flows contain the rootkit: $P(R) = 0.005$, so benign traffic is $P(R^c) = 0.995$.
• True Positive Rate (Sensitivity): If traffic contains the rootkit, the IDS raises an alert with $99\%$ probability: $P(+ \mid R) = 0.99$.
• False Positive Rate: If traffic is benign, the IDS falsely triggers with $2\%$ probability: $P(+ \mid R^c) = 0.02$.

Objective: If an incoming packet flow triggers an IDS alarm ($+$), calculate the posterior probability that the packet actually carries the malicious rootkit ($P(R \mid +)$).

Step 1: Compute Total Probability of an Alarm ($P(+)$)

Using the Law of Total Probability:
$P(+) = P(+ \mid R)P(R) + P(+ \mid R^c)P(R^c)$
Substitute the known values:
$P(+) = (0.99 \times 0.005) + (0.02 \times 0.995)$
$P(+) = 0.00495 + 0.01990 = \mathbf{0.02485}$
Observation: Notice that false alarms ($0.01990$) outnumber true alarms ($0.00495$) by a factor of 4!

Step 2: Apply Bayes' Formula

$P(R \mid +) = \frac{P(+ \mid R)P(R)}{P(+)} = \frac{0.00495}{0.02485} \approx \mathbf{0.1992} \quad (19.92\%)$

Step 3: Analytical Insight

Despite an impressive $99\%$ detection sensitivity and a low $2\%$ false positive rate, an alarm only indicates a $\approx 19.92\%$ chance of an actual breach.
This counterintuitive result is known as the Base Rate Fallacy: because benign traffic is overwhelmingly more common ($99.5\%$) than malicious traffic ($0.5\%$), the sheer volume of benign opportunities generates four times more false positives than true positives.

---

◇References & Navigation

• Previous Topic: ◇Hypothesis Testing
• Next Topic: ◇Random Variable
• Related Notes: ◇Total Probability Rule, ◇Conditional Probability and Multiplicative Rules